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Researchers have long recognized that an increase In statistical precision 
may result from the judicious use of information on relevant antecedent vari- 
ables. Doth the analysis of covariance and the analysis of variance of an 
index of response of the form Y - KX have been advocated as precision gaining 
analyses for a simple random assignment design (Fisher, 1925, 1935). An in- 
crease in precision may also result from the use of the antecedent variable 
as a blocking variable in a randomized block design. Gourlay (1953) compared 
the effectiveness of the analysis of covariance (ANCOVA) and the analysis of 
variance (ANOVA) of indices of response relative to simpJe ANOVA on the dependent 



variable to determine the extent to which precision was improved by use of in- 
formation on the antecedent variable. Cox (1957) examined the same question 
for ANCOVA, ANOVA of correct and incorrect indices of response, and ANOVA of a 
randomized block design. Porter and McSweeney (1970, 1971) examined all four 
parametric procedures — ANCOVA, ANOVA of correct and incorrect indices of re- 
sponse, ANOVA of a randomized block design, and simple ANOVA on the dependent 
variable — and their nonparametric analogues under conditions for which the para- 
metric tests would be optimal. The findings of these simulation studies and the 
earlier studies by Gourlay and Cox apply to data in which: 

(1) the dependent variable, Y, is conditionally normally distributed 
with equal variances for each of the t treatment groups; 

(2) phe regression equations of Y on X for each treatment group are 
linear with equal slopes; 

(3) the errors are independent. 

An analytic study of the robustness of ANCOVA to violation of its 
assumptions (Atiqullah, 1964) demonstrated that use of the ANCOVA model 
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» M + ct , + B(X^j - X) + e^j 

in the presence oC a quadratic component of regression of Y on X leads to 
serious bias in the estimation of the treatment effects and the mean squares 
for t > 2 treatments. Surprisingly, the presence of identical normal distribu- 
tions on the concomitant variable X and a large number of treatment groups — 
remedies the reader might assume would mitigate the effects of bias — are inade- 
quate to overcome the bias in the mean squares for ANCOVA. Cox (1957) observed, 
^'...if the regression is non-linear but smooth, blocking methods will remain ef- 
fective, while covariance methods will not unless the linear component accounts 
for most of the regression..." (p. 157). Sukhatme and Sukhatme's (1954) compari- 
son of two sampling procedures and their associated estimation techniques in a 
sample survey context has implications for the comparison among the previously 
mentioned experimental designs and their respective analyses. Sukhatme and 
Sukhatme examined the precision of the simple regression estimate of the mean 
in a simple random sampling desip-^ relative to the stratum-weighted mean estimate 
in a stratified sample. Such a comparison is analogous to examining the pre- 
cision of ANOVA of an index of response for a simple random assignment design 
relative to ANOVA of a randomized block design. The authors observed: 

"...the stratified sample will, in general, furnish a more 
efficient estimate than the simple regression method. The 
relationship between Y and X is also not alv/ays found to be 
linear in practice, in which case the efficiency of the re- 
gression estimate is further reduced. For, while stratified 
sampling with suitably chosen strata can take care of any type 
of relationship, the regression estimate can eliminate only 
the effects of the linear component of the relationship." (p. 210). 

Various writers on experimental design have observed that when the magni- ' 
tude of the linear correlation between the concomitant variable and the depen- 
dent variable is less than .3, the use of ANCOVA will gain little if any pre- 
cision relative to ANOVA in a simple random assignment design. For linear 
relations such that p < .4, blocking is preferable to covariance analysis, 
while for values of .4 < p < .6, there is no clear preference between ANOVA 



of a randomized block design and ANCOVA of a simple random assignment design. 
When p > .6 ANCOVA is somewhat more precise than ANOVA of a randomized block 
design, and at p > .8, ANCOVA is substantially more precise than ANOVA on 
blocked data. (Cox, 1957; Bancroft, '1968; Elashoff, 1969). The findings of 
Atiqullah, Cox, and the Sukhatmes suggest that in the presence of a substan- 
tial non-linear relationship, ANOVA of a randomized block design may evidence 
equal or greatv r statistical precision than ANCOVA. or ANOVA of an index of 
response. 

Reliance on the questionable robustness to nonlinearity of ANCOVA and 
ANOVA of an index of respc • is one option available to the researcher 
seeking increased statistical precision through the use of a concomitant vari- 
able. Another option consists of selecting a statistical procedure which uses 
the concomitant information but does not require linearity. Randomization 
tests for thij analysis of variance of a randomized block design (Baker and 
Collier, 1968) meet these criteria as does a newly developed randomization test 
for the analysis of covariance (Robinson, 1973). Rather than work with ran- 
domization procedures, we haVe chosen to retain the relative but not the actual 
magnitude of the observations through ranking and the use of nonparametric ^ 
tests based on ranks. A nonparametric test statistic satisfying these re- 
strictions is available as an analogue for each of the parametric procedures 
reviewed. The four nonparametric techniques considered are: 

(1) The Kruskal-Wallis test of equality of mean, ranks for a 
one way ANOVA design (KW) ; 

(2) The Friedman test on ranks in a randomized block design (Fr) ; 

(3) ANOVA of an index of response on mean deviated ranks, d - p d , 
(Nil). Indices with underestimated slopes of .8p (NI2)^ ^ ^ 
and overestimated slopes of 1.2 (NI3) are also examined. 

(4) ANCOVA on ranks (NC) . 
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The first two nonparametric statistics are well-known (c.f» Conover, 1971); 
ANOVA of an index of response ari ANCOVA were constructed by applying 
parametric procedures to the ranks, Monte Carlo studies by the authors 
(1971a, 1971b) verified that the small sample properties of the ANCOVA 
based on applying parametric procedures to ranks were comparable to those 
of Quade's (1967) nonparametric ANCOVA which adjusts the dependent variable 
ranks on the basis of a total sample regression estimate. Because the 
small sample properties of the two nonparametric ANCOVAs were comparable, 
the test based on a direct analogy to parametric procedures was chosen 
for its greater familiarity and ease of computation via standard parametric 
ANCOVA techniques. The index of response on the mean-deviated ranks was 
created to take advantage of the retained degree of freedom when the re- 
gression slope for Y given X can be specified a priori rather than requiring 
estimation from the sample. Our past Monte Carlo studies demonstrate 
that the randomized blocks design analyzed by Friedman's ANOVA is a success- 
ful method for improving power over that for a simple random assignment 
design analyzed by the Kruskal Wallis when the correlation between the 
blocking variable and the dependent variable is greater than .A. When the 
correlation is equal to .4, power is not a relevant dimension for choosing 
between the two designs and when correlation is zero the simple random 
assignment design analyzed by the Kruskal-Wallis test is the more powerful. 
Nonparametric ANCOVA on data in a simple random assignment design is more 
powerful than Friedman's ANOVA on data in a randomized blocks design for 
all values of thr correlation between the concomitant variable and dependent 
variable. Moreover the nonparametric ANCOVA is equal in power to the 
Kruskal-Wallis when p^,, = .0, but becomes progressively more powerful Chan 



the Kruskal-Wallis for increasing values of p • The use of ANOVA on a 
correctly determined index of response proved more powerful than either 
the analysis of covariance or the analysis of variance on indices of res- 
ponse with over-estimated or under-estimated slopes for nonparametric tests • 
Overes timation of the slope impaired the power of the test more seriously 
than did underestimation, and the effects of overes timation became more 
severe as p^^^ increased. 

All of the parametric tests were slightly more powerful than their 
nonparametric counterparts for data which completely satisfied parametric 
assumptions. As the linear correlation p increased, the discrepancy 
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between the power of the parametric and nonparametric tests increased, but 
the relationships among the analyses of covariance and analyses of variance 
on the indices of response for both parametric and nonparametric tests were 
unchanged by increasing Py„. The fact that the relative advantage of the 
parametric tests was slight even when the assumptions necessary for their 
valid use were completely satisfied suggests that little loss, and possibly 
considerable gain, in power will result from the more general use of these 
nonparametric analogues. Since the nonparametric cests require only the 
identity of the marginal distributions of X and the monotonicity of the 
XY relationship, they may well be preferred to the parametric tests when- 
ever there is doubt as to whether the conditional distributions of Y are 
normal, the regressions linear or the variances equal. 

Our previous investigations of the small sample properties of the 
nonparametric statistics and their parametric counterparts were restricted 
to data in close agreement with the parametric assumptions • The single 
exception was our decision to use a normally distributed random concomitant 
variable rather than a fixed concomitant variable. The exception was 



motivated by our concern for the utility of the results since educational 
researchers rarely if ever have data available on a fixed concomitant 
variable measured on an ordinal or ratio scale. The purpose of the present 
study was to replicate our previous investigations of goodness-of-f it and 
small sample power of parametric and nonparametric tests with a data genera- 
tion model that further violated the parametric assumptions while still 
satisfying the nonparametric assumptions. Thus the only restrictions on 
the selection of a data generation model were that it provide observations 
on independent experimental units for two monotonically related variables, 
that the observations on each variable allow rank ordering without ties, 
and that there be no treatment by concomitant variable interaction. Again 
the decision was made to have a random rather than fixed concomitant 
variable. Clearly a wide variety of models satisfy these less restrictive 
nonparametric assumptions. 

The selection of a data generation model is one of the most important 
steps in conducting a Monte Carlo study since the utility of thd subsequent 
results is in large part dependent upon that choice. Given the above restric- 
tions, our selection of a general model was guided by the desire to simulate 
data that were consistent with those likely to be encountered by educational 
researchers. Our choice was the asymptotic regression curve or modified 
exponential defined by the equation 
Y = ct + 6(p)^ . 

The asymptotic regression or modified exponential is a general form of 
growth curve with three parameters, a, B, and p. In our use of this curve, 
the random variables X and Y represent the concomitant variable and the 
dependent variable respectively. When p is restricted to values between 
zero and one and Q is negative, the curve rises from a 4- B at X=0 to the 
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asymptote of a £or large X. A member of Che family of asymptotic regression 
cui*ves satisfying these properties is presented in Figure 1. 

Figure 1 



When a=0, the curve simplifies to the exponential, Y = 6(p) , which describes 
a series that changes by a constant ratio p. Exponential or modified exponen- 
tial (asymptotic regression) growth seems more reasonable to investigate 
than second-degree polynomial growth if we want to simulate that type of 
curvilinearity resulting from the operation of a ceiling effect on the 
dependent variable measure. (Croxton and Cowden, 1960) • 

When the asymptotic regression curve is used to represent the under- 
lying relationship between the concomitant and dependent variable, the full 
model for simulating data is 
Y = a + e(p)^ + yZ 

where a, B, p and y are constant, 

2 

X N( y^, a^) 

and 

Z N(0, 1). 

The model represents a bivariate population with a normal marginal distri- 
bution of X, normal and homoscedastic conditional distributions of Y given 
X, and a negatively skewed marginal distribution of Y. The negatively 
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skewed marginal distribution of Y violates the normality assumption of 
both one way and two way ANOVA, and the monotone increasing curvilinear 
relationship between X and Y violates the linearity assumption made by ANCOVA 
and ANOVA of an index of response. Educational researchers encounter 
similar data when there is a ceiling effect on the dependent variable either 
because of a test which was too easy, or because of a true ceiling effect 
on the latent dependent variable. One example of the latter situation is 
the use of mental age as a dependent variable and chronological age as a 
covariable. 

Further specification of the data generation model required assigning 
specific values to each of the several constants in the equation. We wanted 
an asymptotic regression curve and a distribution of X such that the linear 
correlation between X and Y would be near .7 which we felt represented a 
substantial departure from one. A computer program was written to simulate 
data conforming to the asymptotic regression curve with X normally dis- 
tributed. The output of the program was the Pearson correlation between X 
and Y(p ) and the first four moments of the distribution of Y. Using 
samples of size 10,000 the values of p were systematically varied for a=0, 
6=-l, M =0 and a - 1. Setting a=0 simplified the curve to the exponential, 

X X 

Y = Bp . As p increased from .1 to .9, p increased from .29 to .95. 

Ax 

Although p =• .3 resulted in a desirable value of p = .72, the standardized 

XY 

third and fourth moments for the marginal distribution of Y were -5.94 and 
63.78 respectively. We decided to vary the other parameters in search of 
a similar Pearson correlation but a less substantial deviation from normal 
for the marginal distribution of Y. A value of p = .5 had resulted in 
p = .89 and third and fourth moments of Y equal to - 2.53 and 12.62 respec- 
tively. Varying values of 8 from - 1.0 to - 1.5 did not have a noticeable ef- 
fect on reducing the value of p nor on the third and fourth moments. A lack 



of change in correlation and third and fourth moments was found when \i 
was varied from .0 to 2,0. Increasing a from 1.0 to 1.6 did reduce 
to .72 but resulted in a concomitant increase in negative skew and kurtosis 
to approximately that of the value for p = .3. When we set p = .4, a = 0, 
8 = -1 and p = 0, an increase in o from 1 to 1.3 resulted in a decrease 

X X 

in p^ to .73 with third and fourth moments equal to -A. 98 and 39.03. 

XY 

Therefore we used p ^ .4, a = 0, g = -1.0, y = 0, and = 1.3 

X X 

X 2 
for our data generation model. Thus Y = -.4 + yZ and X N(0, 1.3 ). 

Unfortunately, subsequent runs for the same set of parameters revealed that 

the third and fourth moments of Y were quite unstable for samples of size 

10,000. The implication is that we could have left the standard deviation 

of X at one without markedly changing the third and fourth moments of Y. 

The remaining constant to be defined xa the data generation model was 

Y, the standard deviation of the conditional distribution of Y given X. 

The value of y was set according to the 'desired strength of relationship 

between X and Y as defined by 

2 

^ " ^ a 2 » 

y 

2 

where a 2 = y is the variance of the conditional distribution of Y given 

y • X 

X and a 2 is the variance of the marginal distribution of Y. Since 

y 

2 2 2 2 

a > a , let a = a + C where C > 0. Then 
y y.x y y.x 

2 

n =1 2 

Y + C 
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The standard deviation y can be evaluated in terms of the desired n as 

2 

soon as C is known. Analytic attempts to solve for C = o in the reduced 

y 

model 

Y = -.4^ where X ^ N(0, 1.3^) 

failed. Consequently a simulation based on 100,000 observations from the 

reduced model was used to estimate C = 11.3906. Thus for a given value of 
2 

n , Y was set equal to 




11.3906 



Finally, a pseudo-random unit normal deviate generator for the IBM 370-165 

computer was used to. provide observations on the random variables X and Z 

in the data generation model. The unit normal deviate generation involved 

two stages. First, the multiplicative congruent method was used to generate 

sixteen pseudo-random numbers from a uniform distribution. Second, the 

sixteen numbers were summed and linearly rescaled to provide a pseudo-random 

unit normal deviate via the Central Limit Theorem. 

Design of the Monte Carlo Study 

The parameters of the Monte Carlo study were t, the number of treatments; 

2 

b the number of experimental units in each treatment; and n , the correla- 
tion ratio. Figure 2 presents a summary of the conditions under which the 
sampling distributions of the nonparametric and parametric test statistics 
were investigated. For each X in Figure 2, sampling distributions based 
on 1,000 samples of size tb were g'Hnerated for the null case of no treatment 
effects and for a single noncentral case. Both central and noncentral 
sampling distributions were described by the frequencies of the various test 
statistics falling above their respective critical values for the .10, .05, 
and .01 levels of significance. 



Figure 2 
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t = number of treatments 
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The population distributions indicated in Figure 2 were chosen to 

facilitate comparisons with the results from our earlier studies. Three 

values of t were investigated because of possible trends in the sampling 

distributions as the number of levels of the treatment independent variable 

Increase, The smallest value of t was three since the Wilcoxon matched 

pairs test offers a more powerful alternative to the Friedman test when t 

equals two. Three values of b were investigated because of possible trends 

in the sampling distributions as the number of units under each level of 

the independent variatle increases. The smallest value of b was five 

since exact tests would be more appropriate for smaller values. Three 

2 

values of th- correlation ratio (n = ,8, ,6, .4) were investigated since 
the deviation from parametric assumptions increases with- the size of the 
correlation ratio. The noncentral case was created by adding the value 
-| a to the dependent variable value of each unit under one level of the 

^ y 

treatment independent variable. The choice of •;r a was made because it 

2 y 

seemed to represent a deviation from the null hypothesis that most educa- 
tional researchers would wish to notice. ^Further, it produced intermediate 
values of power which facilitated comparisons of the various test statistics. 
Defining the Indices of Response 

Prior to the actual generation of the sampling distributions, a final 
set of parameters had to be defined. Both the parametric and the nonpara- 
metric indices of response require a priori knowledge of the slope of the 
regression line for predicting the dependent variable from the concomitant 

variable. The slope for the parametric index, 6 , is defined on the 

^ y,x 

original observations while the slope for the nonparametric index, p^, 
is the Spearman correlation. Both parameters were estimated for each of 
the three values of the correlation ratio by generating 4,000 samples with 
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eight treatment groups and eight units per treatment group in each sample. 
The large samples (64) were used so that the estimates of p would be 

8 

minimally affected by sample size. 

The three estimates of each parameter and their standard errors are 

presented in Table 1. The value of ^ was approximately 1.78 for all 

three values of eta squared, but the value of p increased with an increase 

s 

2 2 
in eta squared, i.e., ,3595 for n = ,4 to ,6027 for n = .8. 

Since nonparametric and parametric indices of response require a priori 

information about the parameters p or g , it was also of interest to 

s y.x 

investigate the sampling distribution of the F test for each procedure when 

the a priori information was in error. Somewhat arbitrarily we decided to 

investigate the effect of a priori "guesses** about the slopes that were in 

error by twenty percent, either too high or too low. The slopes for the 

incorrect indices are also presented in Table 1, 

The extensive simulations necessary for estimating p and 3 provided 

s y.x 

interesting additional descriptive information about the data generated by 
the model. The Pearson correlation between X and Y as well as the first 
four moments of the marginal distribution of Y for varying values of eta 
squared are contained in Table 2. The mean was approximately -2.0 for all 
three values of eta squared. The variances are in quite close agreement 
with what x^as predicted using the equation 

0^ = 11,3906 + Y^, 

y 

given earlier. Skewness, kurtosis and Pearson correlation increased with 
increases in eta squared as was expected. Skewness ranged from -1,8721 to 
-5.3158, kurtosis ranged from 10.4755 to 73.8704 and the Pearson Correlation 
ranged from .4436 to .6301. 



TABLE 1 

ESTIMATES OF THE SLOPES FOR DEFINING THE P/\RAMETRIC AND NONPAKAMETRIC 

INDICES OF RESPONSE 
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TABLE 2 



A DESCRIPTION OF THE SIMULATED DATA FOR THE MARGINAL DISTRIUUTION OF Y 
FOR VARYING VALUES OF ETA SQUARED AND SAMPLES OF SIZE 256,000 
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Correlations Within and Across the Treatment Groups for the Simulated Data 

Table 3 reports the empirically generated correlations on the original 

data and on the ranks assigned to the data. Correlations are reported for 

the case of no tr(j^atment effects on the dependent variable (the central case) 

and for the case in which the dependent variable observations in one of the 

treatment groups have been increased by the addition of 1/2 units (the 

noncentral case). The total sample correlation ratio, E(t^^)> is an empirical 

estimate, based on 1000 data sets, of the expected value of ri^ for the design 

defined by the specified values of t, b, and r)^« A comparison of each of 

the tabled values of E(n^) with the respective value of shows that the 

simulation has been quite successful in achieving the desired amount of 

curvilinearity . The largest difference between the empirical estimate of 

the expected value of and the desired parameter is ,0257 for - .6, 

and the average size of the difference is only .0085. The total sample 

Pearson correlation for the underlying model, E(r ) , is an empirical estimate 

xy 

of the expected value of r when the error-free model Y = - .A with 
^ xy 

2 

X N(0, 1.3 ) is used to describe the X Y relationship. The estimated ex- 
pected values of r are very close to the desired measure of linear rela- 
xy 

tionship, p = -7, and are approximately equal across all designs studied. 

These data demonstrate that it was possible to keep the amount of linear 

relationship constant while varying the strength of curvilinear relation. Thus 

as increases, for constant p , curvilinearity increases. 

xy 

The remaining coefficients are also averages obtained over all 1000 data 
sets for their respective designs, and they represent empirical estimates of 
the corresponding mathematical expectations of those correlations in the popu- 
lation. The coefficient E(r^) is an average across 1000 data sets of the 
pooled within treatment group measures of association where 
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and XY^^, represents a sum of squares within groups on the variables X and Y. 

Since it is a mea ce oE the linear relationship of X Y, we find, as expected, 
2 

E(r„) ^ E(ti^). The Spearman correlation, E(r„,,) is similarly defined as an 

W KW 

average across all 1000 data sets of the pooled within group measures of associ- 
ation. However, the scores are ranked separately on both X and Y across the 
total sample prior to the computation of the sums of squares. The only com- 
parable measures of association obtained from the application of Pearson and 
Spearman correlation coefficients to these data are the respective pooled 
within group estimates given by r^' and t^-^^* When empirical estimates of the 
expected values of these quantities are found from the summary results of the 
1000 replications, E(r^) is consistently higher than E(rj^^) by .055 to .075 
units, with no clear pattern of decrease in the discrepancy as a function of 
the total sample size, b t, alone or the value of alone. If the joint dis- 
tribution of X and Y were bivariate normal, we would expect 

Pg = 6 (sin"^(p^Y / 2))/7T 
and E(rg) = 6 ( sin"^ -f (n - 2) sin "*'"(P;j^y2) ) 7 (7r(n + 1) ) . 
(Moran, 1948). Kendall (1949) demonstrated that these relationships may be 
substantially in error for samples from other than a bivariate normal distri- 
bution and that no simple modifications exist to express the exact relationship 

between Pq arid p for arbitrarily specified populations. Kendall's illustra- 
b XY 

tion of a particularly large discrepancy between predicted and actual values of 
p» occurs for a skewed, leptokurtic population not unlike the population defined 
by the dependent variable Y. For our data, use of the formula for pg in terms of 
p^ yields a value of r^^^ = .634 when E(r^)> .6513 (b=8, t=3,n^=-8). This 
value is noticeably different from the obtained value ECr^^^) = .5891. The 
heavy concentration of Y values in the upper tail of the distribution may be 
responsible for the sizable differences between E(r^) and ECr^^^). 
Q The remaining measure of correlation in the central case, E(rg), is the 

average across simulations of the Spearman correlation based on all b t ranks. 
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As expected, E(i^g) increases with an increase in n for fixed b t. The values 

of r^ also increase slightly with an increase in b t, which may be reflective 

of the diminished effect of discreteness. 

The average values of r„ and r„„ for the noncentral case can be compared with 

S RW 

those for the central case. The values of E(r ) are relatively constant over the 

RW 

two cases with discrepancies of (.0012, ,0002, .0018, .0034, ,0005, .0009, and 

.0003) respectively. The total sample values E(r ) are smaller in the noncentral 

case than in the central case. The direction of this difference is expected since 

the addition of values of 1/2 to the dependent variable in one group with no 

compensating change in the values of X, introduces noise into the X Y relationship. 

2 

Goodness-of-Fit and Empirical Power for Varying Values of n When t = 3 and b = 8 
Table 4 compares the empirical sampling distributions -of the test statistics 
for the Kruskal-Wallis (KW) , Friedman (Fr) , parametric ANCOVA on ranks (NC) and 
nonparametric indices of response on mean deviated ranks with the correct slope 
(Nil) and with' slopes that are systematically underestimated (NI2) and overesti- 
mated (NI3). The three rows labelled "central" depict the goodness-of --f it of the 

empirical distributions to their respective null distributions. Both the Kruskal- 

2 

Wallis and Friedman testa are referred to X^^j^, the ANCOVA test refers to bt-t-1* 

and all of the ANOVAs on indices of response use F . /, , n • Since all of the 

t-l,t(b-l) 

sampling distributions were based on 1000 cases, standard errors for the estimated 



actual alphas can be determined from S.E. = /p(l-p)/1000 where p denotes nominal 

alpha level. For nominal alphas of .10, .05, and .01, the respective standard 

errors are .009, ,007, and .003. 

The nominal and empirically estimated actual alphas are in close agreement 

for the nonparametric ANCOVA and the ANOVAs of correct and incorrect indices of 

response. Without exception these empirical alphas are within two standard errors 

2 

of the respective nominal a for all values of n . The Kruskal-Wallis test is 
noticeably conservative for nominal alpha values of .10 and .01 and slightly 
conservative for a nominal alpha of .05. The Friedman test is somewhat 

O 

^J^(^" liberal at a = .10, but its empirical sampling distribution does 



not depart appreciably from good fit to the chi-square distribution at 
the other nominal alpha levels. On the basis of these data it can be con- 
cluded that the fit of the nonparametric analysis of covariance and analysi 

of variance of an index of response to their respective F distributions is 

2 

quite good and is independent of the size of the correlation ratio, n . 

2 

The fit of the Kruskal-Wallis and Friedman tests to ^ is somewhat poorer 

2 

although it too seems independent of the size of n . 

The goodness of fit of the corresponding parametric analogues: 

ANOVA for a one way simple random assignm- at design (Al) ; 

ANOVA for a two way randomized block design (A2) ; 

ANCOVA for a one way simple random assignment design (PC) ; 

ANOVA of an index of response for a correctly determined index 
(PIl) , an underestimated slope (PI2) , and an overestimated 
slope , (PI3), 

to their respective F null distributions can be studied by examining the 

"central*' case reported in the first three rows of Table 5. Fit to the 

respective null distributions is good throughout the table^ with the only 

2 

exceptions occurring for the index of response when n = .8 and a = •01* 
In that case all three ANOVAs of the indices yield slightly conservative 
tests; empirical alphas fall at least tv;o standard errors below the 
nominal alpha = . 01. 

The fit of the test statistics to their corresponding null distri- 
butions is somewhat benter for the parametric than for the nonparametric 
tests when the comparative goodness-of --f it is described by the relative 
incidence of values of estimated actual alpha that fall outsidp the 68 
percent and 95 percent probability intervals defined on nomina^ alphas* 
For the nonparametric analyses, a total of 17 out of 54 values of estimated 
actual alphas fall outside their respective 68 percent probability inter-. 



vals and, of these, 6 values are also outside their corresponding 95 percent 
probability intervals. For the parametric analyses, 13 of 54 values are 
not within the corresponding 68 percent probability intervals and, of 
these, one value falls outside the 95 percent probability interval. 
Although the parametric test statistics seem to exhibit slightly better fit 
than do their nonparametric analogues, the differences in absolute dis- 
crepancy between nominal a and estimated actual a , 

z = [nominal a - estimated actual a| , 
are not large. For nominal a = .10, the mean difference in fit between 
the parametric tests and their nonparametric analogues is - = -.005, 
with the discrepancy almost entirely attributable to the poor fit of the 
Kruskal-Wallis and Friedman tests. The corresponding mean differences for 
a = ,05 and .01 are +.002 and +.001 respectively. The somewhat poorer fit 
of the Kruskal-Wallis and Friedman tests may be attributable to the use 
of the chi-square dis,tribution, the traditional large sample approximation 
for each of these tests, rather than the F distribution, found by Wallace 
(1959) to give slightly better fit. 

Tables 4 and 5 may also be used to compare the small sample power 
against a slippage alternative for the nonparametric and paramtx^tric tests 
included in this study." The values appearing in the three rows labelled 
"noncentral" represent the empirical power of the various tests at a = .10, 
.05, and .01. The maximum standard error of the reported empirical powers, 
.016, occurs for power equal to .5. To facilitate power comparisons, we 
have averaged power differences between statistics across the nominal 
alpha levels. Although the power comparisons at the differing nominal 
alpha levels have slightly different precision, the differences in precision 
do not seem substantial enough to negate the use of these summary measures. 
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When the power of each of the nonparametric procedures which use informa- 
tion on the concomitant variable is compared with that of the Kruskal-Wallis 

test which makes no use of this information, the former is as high or higher for 
2 

all values of n , all nominal alpha levels and all nonparametric tests using the 

antecedent information. The corresponding differences in power relative to the 

2 

Kruskal-Wallis test for n = .4 are .024 for the Friedman test, .049 for nonpara- 
metric ANCOVA, .055 for an ANQVA of a corract index of response, and .054 for both 

the underestimated and overestimated slope in ANQVA of an index of response. When 
2 

n = .6, the respective differences increase to .054, .062, .069, .067 and .064. 

f^' 2 
Further increases are noted for n = .8, where the corresponding differences are 

\ 

.088, .147,-<159, .159 and .144 respectively. 

As expected, the empirical power of nonparametric ANQVA of the correct and 

2 

incorrect indices of response exceeds that of all other statistics when n is 

2 

large. This difference is already noted when n = .4. The slight superiority 

of the indices with respect to ANCQVA was anticipated on the basis of the retained 

degree of freedom in the former technique and its loss in the latter. The superi-- 

2 

ority of the indices of response and ANCQVA to the Friedman test at n = .4 is 
a little surprising if the designs alone are compared. Such a comparison over- 
looks the fact that the Friedman test does not exploit the existence of inter- 
block differences to gain power. Hodges and Lehmann (1964) hypothesized that 
the use of intrablock ranking in the Friedman test, with the attendant disregard 
of interblock differences, resulted in a less than optimal nonparametric test 
for the randomized block design. Their test, based on intra- and interblock com- 
parisons, is asymptotically more efficient than the Friedman test. We conjecture 
that because of the exclusive use of intrablock ranks in the Friedman test, the 
full advantage of blocking may not be gained relative to the use of direct ante- 
cedent variable adjustment in ANCQVA and ANOVA of indices of response. The power 
ANQVA of an index of response using an underestimate slope is comparable to 
that of an index using the correct slope, while that of an index using 
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an overestimated slope is slightly less. The discrepancies in power between 

2 

the under- and over -estimated slope increase with increasing n from .000 
to .003 and .012 respectively. We had found differences comparable to 
these in our earlier work with linear relations. Now, as then, it seems 
reasonable to ascribe increasing differences in the power of NI2 versus 
NI3 to underestimation in the average noncentral values of r^. A compari- 
son of the average noncentral values of the total sample Spearman corre- 
2 

lation for n = .4, .6, and .8 (Table 3) with the estimates of to 

define the incorrect slopes (Table 1) reveals that the underestimated 

slopes, '^Pg* consistently closer to the average noncentral total 

sample p than are the overestimated slopes, 1.2 p . The differences for 
s s 

the underestimated slopes relative to the average noncentral p are 

2 

-.0557, -.0591, -.0688 for n = .A, .6, and .8 respectively. The corres- 
ponding differences for the overestimated slope are .0881, .1275 and .1722 
respectively. The comparison of NI2 and NIB, although based on -Sp^ 
versus 1.2p^, employs sample correlations such that the magnitude of the 
error of underestimation is considerably smaller than that ^f over estima- 
tion. Consequently, the differences in empirical power are consistent 
with the differing size of the errors in estimating the slope. 

A comparison of the empirical power of the nonparametric tests with 
that of their parametric analogues shows higher power for the nonparametric 
analysis of covariance and all nonparametric indices of response, irrespective 
of the value of • Moreover, the advantage of the nonparametric tests 
relative to their parametric analogues increases with an increase in the 
correlation ratio. For the analysis of covariance the average differences 
in power for the nonparametric techniques versus the parametric techniques 
vary from .015 at = .A to .011 at .6 and to .071 at = .8. The 

O 
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differences for the correct and incorrect indices follow a similar pattern 
for increasing values of ri^» In the case of the correct index the values 
are .008, .014, and .095 respectively; for the index based on the under- 
estimated slope, the respective quantities are .008, .011, and .077; and 
for the index based on the overestimated slope they are .006, .019, and 
.115. Since an increase in for constant p in the underlying model implies 
an increase in curvilinearity in the model, the results are consistent in 
demonstrating the relatively superior power of those nonparametric procedures 
which assume only monotonicity of the X Y relationship to parametric pro- 
cedures assuming linearity of the relationship. The comparison of nonpara- 
metric to parametric procedures for the randomized block design tends to 
favor the parametric test slightly, despite the fact that the dependent 
variable distribution is negatively skewed and extremely peaked. The dif- 
ferences in power for the Friedman test relative to the F test for the 
randomized block design are - .007, .001, and - .003 for increasing values 
of fj^ when averaged over the nominal alpha levels. The relative weakness 
of the Friedman test in this context occurs primarily in the upper percentiles 
(1 - a > .95) of the empirical sampling distribution and may be partly at- 
tributable to the discreteness of the sampling distribution of the Friedman 
test. As was noted earlier, the Friedman test does not use the interblock 
differences that are a source of increased precision for the F test in the 
randomized block design. This too may be responsible for the lower power 
relative to the parametric competitor. The Kruskal-Wallis statistic also 
exhibits somewhat lower power than the F test for one way ANOVA, especially 
at the »10 and •Ol alpha levels. The conservativeness of the Kruskal-Wallis 
null distribution at these same alpha levels may explain the reduced power 
relative to the F test. When power differences are averaged over the nominal 
alpha levels, the results for KW - F are - .012, - .006 and ,017 for increasing 
Etyj^ n^^ Only at = .8, is the power of the Kruskal-Wallis test superior F's. 



Although not of direct concern in this study, the empirical powers 

of the parametric tests can be examined to determine the extent to which 

they reflect the effects of curvilinearity . All of the parametric procedures 

which employ antecedent variable information are more powerful than the one 

way analysis of variance F test on a simple random assignment design. The 

following differences in power, averaged over the nominal alpha levels, are 

found when the power of each of the other parametric procedures is compared 

with that of the one way ANOVA F test. The differences are reported as 

three- tup les , with the elements ordered in terms of increasing r\^: 

ANOVA in a randomized block design (.020, .048, .109) 

ANCOVA for data in a simple one way design (.023, .046, .093) 

ANOVA on a correct index of response (.035, .049, .093) 

ANOVA on an index of response with an underestimated slope (.034, .050, .099) 
ANOVA on an index of response with an overestimated slope (.031, .039, .049). 

The results support the earlier conjecture that as becomes large, 
the analysis of variance of a randomized block design, which does not assume 
linearity, becomes more powerful than ANCOVA or ANOVA of response on correct 
or incorrect indices. The differences between the indices of response based 
on correct and incorrect slopes are slight at = .4 or .6 (.001 for the 
correct versus the underestimated slope and .004 and .010 for the correct versus 
the overestimated slope). However, at = '8, the index with the under- 
estimated slope is slightly more powerful than the index with the correct 
slope (.006) and noticeably more powerful than the index with the overestimated 
slope (.050) • 

Goodness-of-Fit and Empirical Power for Varying Values of t When b = 8 and n^= .8 

When the goodness-of-f it of the respective parametric and nonparametric 
tests is studied for increasing numbers of treatment groups (t = 3, 5, 8) and 
constant curvilinear relationship (n^ = .8), no pronounced trends emerge. (Tables 6 

ErJc 
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and 7). The majority of the estimated actual alphas for these statistics 
fall within the 95% probability intervals based on the corresponding nominal 
alphas, with only five exceptions out of 54 cases for the nonpararaetric tests 
and fourteen out of 54 for the parametric tests. The fit is similar when 
68% probability intervals are examined: 22 nonparametric estimated actual 
alphas and 31 parametric empirical alphas fall outside of the respective 68% 
intervals. Most of the instances of lack of good fit result in the over- 
estimation of the actual alpha through the use of the nominal alpha. Thus 
the associated tests are conservative. The fit of the nonparametric tests 
is somewhat better than that of the parametric procedures. The parametric 
tests are especially conservative for t = 8; however, it seems unlikely that 
the conservative behavior of almost all the parametric procedures for this 
design is a consequence of the change in design from t = 3 and 5 to t = 8. 
Unreported portions of this simulation study examined the central distribution 
of the parametric tests for varying t and = .6 and did not find comparably 
conservative behavior. Thus it seems more reasonable to assume that these 
results are an artifact of the particular 1000 data sets simulated for t = 8, 
b = 8, = .8 that are used for the analysis of all the parametric and non- 
parametric tests having these design dimensions. 

When the empirical power of the nonparametric tests is examined as a 
function of an increasing number of treatment groups, the actual magnitude of 
the power decreases but the relative ordering among the tests remains unchanged. 
The decrease in empirical power with increased t merely reflects the fact that 
the single differing treatment group makes up a smaller proportion of the 
total sample size (1/3 versus 1/5 versus 1/8 for t = 3, 5, 8 respectively) • 
The power of each of the other nonparametric tests using information on 
the concomitant variable is compared with that of the Kruskal-Wallis test 
and differences are averaged over the nominal alpha levels to yield the 



following results for increasing values of t: 

Fr - KW Power: (.088, .114, .104) 

NC - KW Power: (.147, .151, .153) 

Nil - KW Power: (.159, .159, .153) 

NI2 - KW Power: (.159, .156, .154) 

NI3 - KW Power: (.147, .148, .141) . 

The most powerful of the nonparametric techniques for .8 are the analysis 

of variance on a correct index of response (Nil) and on an index of response 

with an underestimated slope (NI2) . These are followed closely in power 

by the analysis of covariance (NC) and the analysis of variance on an index 

of response with an overestimated slope (NI3) . The Friedman test (Fr) has 

substantially more power than the Kruskal-Wallis test in the presence of 

such a strong X Y relationship, but its power is considerably lower than 

that of the other procedures which incorporate information about the nature 

of the X Y relationship more directly into their respective test statistics. 

A comparison of these nonparametric tests with their parametric 

analogues for the substantial curvilinear relationship implied by p = .7 and 

= .8 reveals that all of the nonparametric tests are more powerful than 

their parametric counterparts for all values of t. The gains in power for 

the Kruskal-Wallis and Friedman tests relative to the F tests for one way • 

ANOVA and ANOVA of a randomized block design respectively are quite slight, 

while those for the other procedures versus the parametric tests demanding 

linearity are somewhat more substantial. For , increasing values of t 

KW - Al Power: (.017, .034, .020) 

Fr - A2 Power: (.013, .013, .029) 

NC - PC Power: (.073, .084, .107) 

Nil - PIl Power: (.095, .102, .117) 

NI2 - PI2 Power: (.077, .093, .085) 

NI3 - PI3 Power: (.115, .117, .128) 

The strong showing of the nonparametric analysis of variance on an index of 

response with an overestimated slope is probably reflective of the very 
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conservative behavior of the parametric analogue in the central case, and 
the reuse of the same data with 1/2 added to one treatment group to 
create the noncentral case. A very slight increase in the advantage of the 
nonparaiuetric procedures to the parametric tests occurs with increasing t. 
Such a finding is consistent with the results of many Monte Carlo studies 
which have established that nonparametric tests tend to make their best 
showing relative to the parametric analogues when differences between populations 
are small and power is low to moderate. 

Goodness-of-Fit and Empirical Power for Varying Values of b When t = 3 and = *8 

Tables 8 and 9 provide evidence with respect to the goodness-of-f it of 
the nonparametric and parametric tests for increasing numbers of observations 
per treatment group. Both tables indicate relatively good fit of the empiri- 
cal sampling distributions of the test statistics to their corresponding null 
distributions. Only 6 of 54 estimated actual alphas for the nonparametric tests 
and 3 of 54 for the parametric tests fall outside the 95% probability intervals 
based on the corresponding nominal alphas. The fit is similar when 68% proba- 
bility intervals are studied: 11 nonparametric and 18 parametric estimated 
actual alphas fall outside the respective 68% intervals. The quality of the 
fit does not vary appreciably with b, although there is slightly better fit 
for the nonparametric tests at larger values of b. The Kruskal-Wallis test 
and the Friedman test are once again the principal source of lack of fit among 
the set of nonparametric tests examined. All 5 estimated actual alphas that 
are outside the limits of the corresponding 95% probability intervals on the 
nominal alphas are identified with either the Kruskal-Wallis or the Friedman 
test. 

The power of all of the parametric and nonparametric tests considered is 
a monotonically increasing function of the sample size per treatment group, b. 
Q Substantial increases in the power of the tests occur as b increases from 
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moderately affected by changes in b. 

Each of the nonparametric tests using information on the concomitant 
variable is more powerful than the Kruskal-Wallis test which ignores this 
information. Furthermore, the gain in power relative to the Kruskal-Wallis 
test increases with an increase in the number of observations. The analyses 
of variance on a correct index of response and on an index with an under- 
estimated slope are again slightly higher in power than the analysis of 
covariance or the analysis of variance on an index of response with an over- 
estimated slope and substantially more powerful than the Friedman test. The 
resulting differences in power relative to the Kruskal-Wallis test, averaged 
over the nominal alpha levels, are: 

Fr - KW Power: (.032, .088, .109) , For, increasing .values of b 
NC - KW Power: (.086, .147, .186) 
Nil - KW Power: (.097, .159, .192) 
NI2 - KW Power: (.097, .159, .185) 
NI3 - KW Power: (.085, .147, .183) . 

All of the nonparametric tests except the Friedman test are more power- 
ful than their parametric counterparts. The Friedman test, which exhibits a 
conservative null distribution, is slightly less powerful than the F test for 
a randomized block design. The Kruskal-Wallis test realizes only a slight 
advantage in power relative to the F test for one way analysis of variance, 
but this advantage increases with an increase in the number of observations 
per group. The nonparametric analysis of covariance and the analyses of 
variance of the indices of response are all substantially more powerful than 
their parametric analogues, and this gain in power increases with increasing b. 
These results are indicative of the effect of curvilinearity in decreasing 
the power of the parametric tests which assume linearity. Our earlier work 
dealing with, similar design dimensions but a data model satisfying parametric- 
assumptions provides an informative comparison with these data employing a 
curvilinear relationship. 



29 



Empirical Power of the Nonparametric Tests Minus That of Their 
Parametric Analogues Requiring Linearity, Averaged Over a Levels 
for t = 3, b = 5, 8, 10 and Differing Assumed X Y Relationships 



Parametric Assumptions Met Curvl linearity Present 





P = 


.6 




p = .7 




.8 


NC - PC 


- .033 


- .043 


- .053 


.041 


.071 


.112 


Nil - PIl 


- .050 


- .058 


- .058 


.056 


.095 


.128 


NI2 - PI2 


- .022 


- .050 


- .049 


.053 


.077 


.110 


NI3 - PI3 


- .068 


- .077 


- .075 


.055 


.115 


.154 



As the results indicate, the advantage of the parametric tests increases 
with increasing b when parametric assumptions are met, but the disadvantage 
of the parametric tests increases when the X Y relationship is markedly 
curvilinear. 

Summary of the Empirical Results 

The data for this study indicate some consequences, in terms of the 
estimated Type I Error and power, of the choice between a nonparametric 
test and a parametric test when there is a nonlinear relation between the 
dependent variable and the concomitant variable. The empirical sampling 
distributions of all of the parametric and nonparametric tests employing 
the F distribution showed relatively good fit to their respective null 
distributions. The Kruskal-Wallis and -Friedman tests, both of which were 
referred to the chi-square distribution, occasionally demonstrated poor fit. 
The Kruskal-Wallis test was consistently somewhat conservative, while the 
Friedman test tended to be conccrvative in the extreme tail of the distri- 
bution but liberal at the a = .10 level. 

All of the nonparametric tests were more powerful than the Kruskal- 
Wallis test at all levels of relationship studied and for all design variations 
of t and b. Among the nonparametric tests using information on a concomitant 
variable, the tests could be ranked in order of decreasing empirical power as: 
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ANOVA of correct 1, ANOVA of index > ANCOVA ^ ANOVA of index > Friedman 
index of response slope too low slope too high test 

The differences between an index of response based on a correct slope and one 
based on an underestimated slope were typically very slight, as were the dif- 
ferences in power between analysis of covariance and analysis of variance of 
an index of response on an overestimated slope. 

When the nonparametric tests were compared with those parametric tests 
which demand linearity of the X Y relation, the relative superiority of the 
nonparametric tests increased with increases in the size of ri^> the number of 
observations per group, b, and the number of treatments, t. The effect of 
skewed, leptokurtic distributions on the dependent variable was not sufficiently 
strong to yield nonparametric tests for the one way ANOVA (KW) and for the ran- 
domized block design (Fr) which were consistently more powerful than their para- 
metric analogues. However, for = -8, the Kruskal-Wallis and Friedman tests 
tended to be slightly more powerful than their parametric counterparts. 

When the parametric tests were compared among themselves, our conjecture 
that a sizable amount of nonlinearity might favor the F test for the randomized 
block design over its parametric counterparts which require a linear X Y rela- 
tion was substantiated. Thus conclusions regarding the decreasing power of 
parametric procedures for data in which the X Y relation is linear, 

ANOVA of correct > ANCOVA > ANOVA of a randomized 
index of response block design 

are not supported in the nonlinear case. Instead, 

ANOVA of a randomized > ANOVA of a correct > ANCOVA 
block design index of response 

The study as a whole supports the viability of selecting a nonparametric 
test in preference to a parametric one if the relationship between the depen- 
dent variable and the concomitant variable is nonlinear. The declared chances 
of committing a Type I Error adequately describe the actual chances, and the 
power of the nonparametric analyses of variance of indices of response and 
of analysis of covariance is higher than that of their parametric analogues. 



^\'^e have asserted that, "Since an increase in for constant p in the under- 
lying model implies an increase in curvilinear ity in the model,..." This 
statement does not imply that riband p are reported for comparable data. 
The value of p pertains to the linear correlation in the error-free model, 

Y = - .4^ with X N(0, 1.3^) . 

The value (1 - p^) is a measure of the lack of fit of a linear X Y relationship 
to the function specified by 

Y = - .4^. 

The correlation ratio 

2 1 
n = 1 - y .X 

y 

is based on the full model which includes normally distributed error, 

Y = - .4^ -f yZ with Z N(0, 1) . 

The value (1 - n^) -^^ / is a measure of "pure error" in fitting 

empirical data to the^nonline^r functional X Y relation. Unfortunately, no 
comparable data are reported for the total sample values of the Pearson 
correlations. These would measure both "pure error" and lack of fit (Draper 
and Smith, 1966). Of necessity, comparisons will be made between E(n^) and 
E(r^), both computed on the full model, but the former measuring association 
in the total sample and the latter measuring the pooled within groups linear 
correlation. Under the null hypothesis of no treatment effects and the as- 
sumption of identical distributions on the concomitant variable, needed by 
the nonparametric tests, r^ and r for the total sample both serve as estima- 
tors of the corresponding population Pearson correlation. It can be noted 
that as increases, both E(^^) and E(r^) increase, but ^t differing rates. 
Since r^ < x]^ in the presence of curvilinearity and r^ = for linearity, 
the increasing values of E(ri^) - (E(r ))^ with an increase in imply an 
increase in curvilinearity. 
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TABLE 3 



TOTAL SAMPLE GORRKLATION RATIO ECn^), TOTAL S/VMPLE PEARSON CORRELATION FOR 
THE UNDERLYING MODEL E(r ) , AND AVERAGE CORREI^TIONS WITHIN TREATMENT 
GROUPS E(r„), ACROSS ALL RANKED OBSERVATIONS E(rg) AND WITHIN TREAT- 

>IENT GROUPS ON RANKED DATA ^(i:^^) 



CENTRAL 



NON-CENTRAL 



n 

t 

b 



= 


.1, 


.6 


.8 




- 


3 


3 


3 


5 


8 




8 


8 


5 


8 


10 


8 


8 


E(n^) 


. 39?.0 


.57/.3 


. ?029 


.8010 


.8135 


.8018 


. 8068 




. 7109 


.7175 


.6978 


.6925 


. 0JS7 


.6876 


.70/, 8 


E(r^.) 


■ .A1S7 


.5261 


. 6/4/1 5 


.6513 


.o5l6 . 


.6592 


.6553 


E(r3) 


.3539 




. 5698 


. 5910 


. .5-393 


.6038 


.60/^ 




.3505 


.U99 


. 5670 


. 5891 


. 5893 


.6031 


,6019 


ECrg) 


.3433 




= 5353 


.5510 


.55;:2 


. 5772 


. 5S87 




.3517 


, 7+501 


. :6S8 


. 5857 


. 5898 


.60:',? 


. 6o;:2 
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